The present invention relates to computing, and in particular, to a systems and methods for creating a data structure in which objects may be added and removed according to established procedures.
Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.
FIG. 1 is a generic depiction of a conventional server which is configured to respond to a plurality of requests. In particular, a processor 100 of the server receives requests (a, b, c, d) 102 from one more sources (S1, S2, S3, S4) 104. The requests can exhibit some overlap 103 in time (for example requests c and d) or be separated 105 in time (for example requests c and b).
The requests are then packed into a queue 106 according to a queuing procedure 107, and processed in that order by the processor. The results are placed in a result set 108 or are simply returned. For the purposes of FIG. 1 there are no different service levels for the requests.
Typically, the server processes the requests in a queuing procedure according to First-In First-Out (FIFO) principle. Under such a mode of operation, requests are inserted into the queue by the processor in the order they were originally received. Thus, this mode of operation is based exclusively on time, independent of any other criteria, such as, priority, urgency, relative importance, or customer service level.
Increasingly, however, a service is expected to be configured to receive requests and provide responses prioritized according to different levels of service. Examples of such service levels may include free, basic, professional, and premium.
More rapid response times would be expected for correspondingly increasing levels of service. In one implementation a priority number is assigned to a request and the requests of the highest priority are removed from the queue first. However, this can lead to two potential problems.
A first possible problem is starvation. In particular, higher priority requests may dominate the processor and the lower priority requests are neglected.
A second possible problem is a lack of guaranteed level of service. In particular, the lower priority requests could have their services quality go up and down unpredictably and/or drop below a previously agreed upon level when higher level request come in.
Currently, these issues may be addressed by hosting a different cluster per service level. This approach, however, may not result in the efficient utilization of available processing resources. For example, at times where a particular service level is undersubscribed, a cluster may be operating at less than full capacity. As well, additional licensing fees may be owing.
The present disclosure addresses these and other issues with systems and methods allowing server performance (e.g. response time) to be tailored for a particular service level.